Active Advice Seeking for Inverse Reinforcement Learning
نویسندگان
چکیده
Intelligent systems that interact with humans typically require input in the form of demonstrations and/or advice for optimal decision making. In more traditional systems, such interactions require detailed and tedious effort on the part of the human expert. Alternatively, active learning systems allow for incremental acquisition of the demonstrations from the human expert where the learning system generates the queries. However, active learning allows for only labeled examples as input, significantly restricting the interaction between expert and learning algorithm. Advice-based learning systems increase the expressiveness of the interaction, but typically require all the advice about the domain in advance. By combining active learning and advice-based learning, we consider the problem of actively soliciting human advice. We present the algorithm in an inverse reinforcement learning setting where the utilities are learned from demonstrations. We show empirically the contribution of a more expressive advice over traditional active learning approaches.
منابع مشابه
Inverse Reinforcement Learning via Ranked and Failed Demonstrations
In many robotics applications, applying reinforcement learning (RL) can be especially difficult, as it depends on the prespecification of a reward function over the environment’s states, which is often hard to define. Inverse Reinforcement Learning (IRL) [1] attempts to address this problem, by utilizing human demonstrations to learn the reward function, without having a human explicitly define...
متن کاملActive Learning from Critiques via Bayesian Inverse Reinforcement Learning
Learning from demonstration algorithms, such as Inverse Reinforcement Learning, aim to provide a natural mechanism for programming robots, but can often require a prohibitive number of demonstrations to capture important subtleties of a task. Rather than requesting additional demonstrations blindly, active learning methods leverage uncertainty to query the user for action labels at states with ...
متن کاملMulti-class Generalized Binary Search for Active Inverse Reinforcement Learning
This paper addresses the problem of learning a task from demonstration. We adopt the framework of inverse reinforcement learning, where tasks are represented in the form of a reward function. Our contribution is a novel active learning algorithm that enables the learning agent to query the expert for more informative demonstrations, thus leading to more sampleefficient learning. For this novel ...
متن کاملActive Learning for Reward Estimation in Inverse Reinforcement Learning
Inverse reinforcement learning addresses the general problem of recovering a reward function from samples of a policy provided by an expert/demonstrator. In this paper, we introduce active learning for inverse reinforcement learning. We propose an algorithm that allows the agent to query the demonstrator for samples at specific states, instead of relying only on samples provided at “arbitrary” ...
متن کاملProbabilistic Logic Learning via Active Advice Seeking
Machine learning approaches that utilize human experts combine domain experience with data to generate novel knowledge. Unfortunately, most methods either provide only a limited form of communication with the human expert and/or are overly reliant on the human expert to specify their knowledge upfront. Thus, the expert is unable to understand what the system could learn without their involvemen...
متن کامل